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1. INTRODUCTION 

Maximizing data flow is critical in networked control systems (NCS) with minimal resource 
capacity where several control loops share a common network [1]-[3]. Limited network capacity can result in 
packet drop or delay. To eliminate delay or packet drop, which may destabilize or adversely affect the 
response of the NCS, network resources must be used efficiently [4], [5]. Consequently, optimization of 
resource allocation is a major issue in NCS [6], [7]. 

Effective optimization and resource allocation require a good mathematical model of the NCS. In a 
recent paper [8], Ge et al. investigated a Markov model and stochastic control strategy for NCS dynamics. 
Hidden markov model (HMM) of NCS was first proposed in [9]. The authors proposed a stochastic optimal 
controller design for an NCS with network-induced delay governed by an underlying Markov chain of 
unknown probability distribution. Using the Baum-Welch algorithm, they obtained the initial distribution and 
state-transition probability matrix. They assumed three states for the NCS network load: low, medium, and 
high. Using a sequence of observations, they identified the Markov model and used it to design a stochastic 
optimal state feedback controller for the NCS. Nilsson also modeled network states as low, medium and high 
based on network load with transition between states governed by a Markov Chain [10]. The states were not 
directly observable and an HMM model was used for the network with the NCS treated as a jump linear 
system. Based on the HMM, he designed an LQG optimal controller to assure system stability. W. Lim, et al. 
[11], the authors proposed a dynamic bandwidth allocation strategy for orthogonal frequency division 
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multiple access passive optical networks (OFDMA-PONs). Given the dynamic and stochastic nature of 
network, they used an HMM to predict traffic states. They defined the traffic states with two variables, the 
mean and the contrast of the bandwidth request observations. The latter characterizes the degree of traffic 
variation. Dainotti et al. [12] also classified packet-level traffic based on HMM. They used several types of 
traffic data and estimates of packet size and inter-packet time to evaluate their approach. 

Li et al. [13] used fuzzy logic to construct a bandwidth manager. They determined the lower and 
upper bounds for the assignable bandwidth using linear matrix inequalities and the resource constraints. 
Fuzzy logic provides a flexible feedback mechanism to manage NCS that accounts for an upper bound and a 
lower bound on the sampling period. The proposed approach saves bandwidth and improves overall control 
performance as compared to fixed bandwidth allocation. 

Although a rich literature on bandwidth allocation exist, to our knowledge the problem of bandwidth 
allocation for a network shared by NCS and random network traffic was only addressed in an earlier paper by 
[14]. G. Cetin, et al. [14], prediction of future traffic used a conditional poisson model. Although this 
provided a basis for bandwidth allocation, the results can be significantly improved by improving the quality 
of traffic prediction. 

This paper builds on our earlier work to provide better bandwidth allocation for a network with 
random channels. Multiple links including random traffic, time-driven traffic, and event-driven traffic 
sharing a bandwidth is assumed. Network simulation data is used to formulate an HMM model for the 
network. Unlike earlier work, the model has a large number of discrete states that provide a more accurate 
network representation. Using the current traffic of the random network, the traffic at the next sampling point 
is predicted utilizing the HMM. Bandwidth of the network is allocated depending on the resource demands of 
each individual channel by solving a convex l4, optimization problem based on the predicted traffic for the 
random channel and the known deterministic traffic. Although it is assumed that network capacity is time- 
invariant over the planning horizon of the control system, if the capacity is known a priori, the proposed 
method is valid if the capacity changes with time. 

With limited network capacity, resource demand can exceed capacity. This may cause packet 
dropout and/or delay. As in [14], 4 minimization with time-varying sampling period for the time-driven 
channel is used to allocate bandwidth for the random and event-driven channels optimally. If insufficient 
bandwidth is allocated for the time driven sensor channel. The sampling period for the time-driven channel is 
increased to reduce traffic. However, the upper bound on the sampling period should be determined to 
achieve sufficient stability and performance in the closed-loop NCS system [15], [16]. We derive a condition 
for the stability of a linear time-invariant plant with a bounded sampling period using the results of [17]. The 
main contributions of the paper are; 

a. The paper proposes bandwidth allocation method for a network that serves random and deterministic 
traffic. 

b. The paper provides a new adaptive HMM network model with multiple states that can be used for traffic 
prediction. 

c. The HMM model is based on a sliding data window to allow for time-varying traffic conditions. 

d. The paper presents a new stability condition for a linear time-invariant system with time-varying 
sampling period. 

The remainder of the paper is organized as follows: section 2 describes the HMM model for random 
data traffic to predict states using Viterbi algorithm. In section 3, the convex l, optimization problem and the 
dynamic sampling problem are described. Section 4 provides simulation results and discussion. Section 5 is 
the conclusion. 


2. HMM OF NETWORK TRAFFIC 

HMM is a statistical Markov models whose states are not directly observable. Instead, an output that 
is stochastically dependent on the hidden states is directly observable. The observed outputs are related to the 
HMM states by observation emission probabilities. Transition between states is random and is governed by 
state transition probabilities which indicate the probability of going one state to another. A states sequence 
from an initial state to end state can be generated by switching from one state to another based on the 
transition probabilities. The observable sequence of outputs is related to the state sequence by the emission 
probability distribution [18]. 

To obtain a HMM of a network, random network traffic level is allocated to five possible classes, 
which are "very low(VL)", "low(L)", *medium(M)", *high(H)", “very high(VH)". These classes can be 
determined based on total network capacity and network characteristics. With no loss of generality, the 
classes are identified with integers and are expressed as c = (VL, L, M, H, VH} = (1, 2,3, 4, 5}. 
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Assuming that a NCS shares a common network with N nodes that generate random traffic, let t; (k) 
pac/sec be the traffic of the it” random channel at time k, i = 1,2, ..., N. 
The traffic conditions are defined as follows; 
Channel conditions for i" random nodes: 


ar s ed ue bassi (1) 


where ay, is the lower bound for class c and a, is the upper bound for class c for the it random channel. 
For five different classes and N random channels, we need 5" HMM states. For example, consider the case 
of 3 random channels, i.e. N = 3, we use the notation 

state 1 = (11 1] represents (VL VL VL}, 

state 125 — (5 5 5) represents (VH VH VH) and so on. 

The state-transition matrix, whose entries are transition probabilities from one state to the next, can be 
determined by using relative frequency if enough data is collected. 

The emission matrix determines the probability of network congestion based on HMM states given measured 
outputs. The emission matrix is classified based on total random traffic as 
^no congestion very low traf fic (NCVLT)”, 

“no congestion low traf fic(NCLT)”, 

"critically congested(CC)”, “high congestion(HC)” , 

“very high congestion(VHC)” 
r = {NCVLT,NCLT,CC,HC,VHC}.= (1,2, 3, 4,5] 
Let the random traffic be given by 


N 
Be) =) (09 
i-1 
N N N 
B.=) af, B» ab, Bs= > aff, (2) 
i=1 i=1 i=1 
N N N 
gi X alt Bs z 2 all, Be 2 2 
i=1 i=1 i 


where B; is the total random traffic, and B;,i = 1, ..., 6, are traffic bounds for the five network classes. 


If B4 € Ber < B, congestion status r = 1 
If B; € Bir < Bz congestion status r = 2 
If B4 € B,r < B, congestion status r = 3 (3) 
If B4 € Ber < Bs congestion status r = 4 
If B; € Ber < Bg congestion status r = 5 


Based on this classification, we can define the emission matrix to include the probability of congestion states 
according to the HMM states. Since we have 125 states and 5 observation states, the size of the E will 
be 5 x 125. 

The HMM can be defined as A, = (N,M,T,E,7) where T is the state-transition matrix, E is the 
emission matrix, N is the number of the states of HMM (traffic conditions), M is the number of observation 
states(congestion conditions) and n is the window size [9]. The HMM model is shown in Figure 1. In the 
figure, we have; 


dx € 0 = [0j] is the hidden state of the Markov chain at time k = 1,..,n, 0 is 
the hidden state, 0; = i, i = 1,2, ..., N 

O, € O = [oj] is the observation at time k, o; = i,t = 1,2, ..., M 

T = | T4, T; .... tty] are the initial probabilities of the states 

where m; = P(qy = 0j) where i = 1,..., N and k = 1,..,m. 


T (dk dk+1) = [ai jl is the state-transition matrix with 
aij = P(qk+1 = Ojla = 0; 1 Sij SN 
E = le; jl is the emission matrix with 

ej; = P(0,, — 0;|qx = 0; ) 
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T (41, 92) T (42,93) T(Qn-1,49n) 


E (qn, qu) 





Figure 1. HMM model [10] 


To find T for window size n, we first determine the traffic classes from the available data using (1). 
Then we find the transitions probabilities using the relative frequency of state transitions. To find the 
emission matrix E for window size n, we use the ranges of (3). 

Since network dynamics change over time, we determine the adaptive transition matrix and the 
emission probability matrix based on random network traffic data over a specified time window. Using n 
Observation 04, 02 .... 04, of network traffic conditions data ending at the current time k, we find the adaptive 
transition matrix and the emission probability matrix for window size n. The Viterbi algorithm [18-22] is 
used to predict the most likely states at time k = 1,..,n + 1, we save the predicted state at k = n + 1. Then 
we shift the window by one step, and update the adaptive transition and the emission probability matrices. 
The Viterbi algorithm predicts the most likely state at time k = n + 1,..., 2n +1. We save the predicted 
state at k = 2n + 1 and the process is repeated to predict future states. 

The Viterbi algorithm works as follows. Given the HMM model and a sequence of observations, we 
to estimate the most likely sequence of states [18]. The most likely sequence ending with qg = @ is obtained 
using the utility function 


vj = max log P(O, dx) (4) 
qy-0 


The paths that lead to each state from the possible prior states are explored to find the path that has the 
highest probability. For each possible new state, only the highest probability path is saved using Ve 


Vi = ar gmax vë + log (P4, 1g, (9'19)) + log (Po, ilar, (Ox+110)) (5) 


where @’ is posterior state at time k + 1, and Vj‘** is the maximum cumulative log-probability achieved for 


(k + 1)*” time step from k'^ time step. 

Using (5) recursively, we determine the most likely paths from k = 1 to k = n + 1 for window size 
n and save the most likely transitions for backtracking iteration. The final posterior probability of the all 
sequence of states can be maximized by recursively maximizing the joint probability for each possible new 
state. The best final state at time k = n + 1 for window size n can is calculated as 


gn*i- argmax Va: (6) 


Later backtracking iteration is used to find all hidden states in the window (traffic conditions) 


0^ yv k=nn—-1,..,1 (7) 


3. BANDWIDTH ALLOCATION 

Figure 2 shows general structure of NCS. Random nodes occupy bandwith of the network based on 
its network traffic, sensor, controller and actuator nodes are interconnected via network for data transmission 
as in shown in Figure 2. Assume n observations are known initially. Traffic conditions c for each random 
channel at time k + 1 are predicted utilizing the HMM model and the Viterbi algorithm. With knowledge 
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of c, we can assign random traffic for each random channel at time k + 1 by assigning the upper bound a; 
related to the traffic condition to guarantee the required bandwidth allocation for the time-driven sensor node. 

Since the total network bandwidth capacity is constrained, bandwidth is allocated optimally 
depending on the traffic of each channels. The Viterbi algorithm predicts the random traffic at time k + 1 
given the traffic of the event-driven channels. Bandwidth is allocated for the random and event-driven 
channel based on the following optimization as in [14]. 


i — 
Sensor Node Actuator 
Node 


Ad 






Network 


Controller Random 
Node Nodes 


Figure 2. NCS structure [14] 


The sampling period of the NCS is varied to optimize bandwidth allocation. Even for a linear time- 
invariant plant, the closed-loop NCS is not time-invariant and the placement of its eigenvalues inside the unit 
circle is no longer sufficient for stability. Zhou and Zhao provided the following result regarding the stability 
of the perturbed discrete linear time-varying system with state matrix A(k) and perturbation matrix AA (k) 
[17]. 


x(k + 1) = (A(k) + AA(k))x(k), =k =0,1,2,... (8) 


where A (k), AA(k) € "*n 

Theorem 1 [17]: 

Assume that the nominal discrete linear time-varying system with state matrix A(k) is uniformly 
exponentially stable. If there exist a positive constant fj and a sufficiently small constant e such that AA(k) 
satisfies 


Y IIAAGDII S ek + B, k =1,2,..., (9) 


then, the perturbed discrete linear time-varying system system (8) is uniformly exponentially stable. 

Based on the above theorem, we obtain a condition for the stability of a linear time-invariant NCS 

with time-varying sampling period. We assume that the state matrix has geometric multiplicity equal to its 
algebraic multiplicity. 
Theorem 2 : If the linear time-invariant system with digital control is closed-loop stable with state feedback 
control gain K and sampling period for the time driven channel of the related control loop A*, then the system 
is uniformly exponentially stable with time-varying sampling period in a bounded interval 
[Ancsmin’ hycsmax] including h. 


Proof: 
The discrete-time state space model is given by 
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x(k + 1) = [e^ 0? — B (ht (k))]x(k) (10) 


The state-transition matrix and the input matrix can be written as 


n 
An (k) — > 7, oñiht(k) 
l 
i=1 (11) 
UM z;B e^ 094] 


B(ht qo) = SE 


For a zero eigenvalue, we have the term Z;Bh' (k), which can be handled similarly to what we show below. 
Perturbation in the sampling period will result in the perturbation matrix 


AA(k) = » (z: - =) [ea — eAih' G0) (12) 


t=1 


For a sampling period varying in a bounded interval [Tmin, Tmax |, we have 





e^i^ = e^i (9| « |e^icsmas = eg ^i NCSmin Sky = Lem (13) 

Because the nominal dynamics are stable 
BK 

Iz -F < k5,1 = 1, Gg d 
The condition of Theorem-1 [17] becomes 

k-1 

2. I4AQI < ek, e = nk,k; (14) 

j=0 


4. RESULTS AND DISCUSSIONS 
A DC motor position control example adapted from [23, 24] is simulated to compare our results 
with fixed bandwidth allocation. The DC motor transfer function is; 





1000 
G(s) B S(s+1) 
A feedback controller is designed to achieve desired transient response. An upper bound for the 
sampling period of the time-driven channels hi ycesmay> is necessary for obtaining satisfactory NCS 


performance and is calculated based on the system dynamics. The desired response of the system must have 
percentage overshoot less than 20% and settling time less than 0.2 s. For 20% overshoot, required damping 
ratio is approximately 0.46. The natural frequency of the closed loop system is calculated based on the 
specifications as WwW, = 48.73rad/s. The undamped natural frequency should be slower than the packet arrival 
to the controller, to obtain the required sampling frequency for the NCS, «c, is selected as w, = aw, where 


oo scale factor selected to reduce aliasing. Assuming T = 1s, we selected æ = 15 to obtain h; ee sies 
TU 


m = 11 ms. since T for the network is 0.1 s, Ri yesmay = 1.1 ms. Based on hardware limitation on sampling 


rate, we select h; Mw. = 0.6 ms. 

For the simulation, frequency-division multiple access (FDMA) network channel is used in NCS as 
in [25]. The FDMA network channel has a capacity of 495 Mbps and packet size is 100 bytes. Thus, the total 
network capacity is 495 pac/0.1s. The traffic in the three random channels is predicted and the sensor and the 


controller channels generate 100pac/0.1s to satisfy required transients response. 


Optimal resource allocation in networked control systems using viterbi algorithm (Gókhan Cetin) 


1530 O ISSN: 2302-9285 


Random networks occupy 80% of the bandwidth of the network while time-driven sensor and event- 
driven controller nodes occupy the remaining 20%. Fixed sampling period of the time-driven sensor channel 
is choosen as 1ms. If the allocated bandwidth for random channels is high, the allocated bandwidth for the 
controller and the sensor channels may become insufficient and this may cause delay or package drop which 
deteriorates the performance of the NCS as shown in Figure 3. 





Position 
mu 








Time 


Figure 3. NCS output without bandwidth allocation 


Assume that, three random nodes which generate random traffic and two deterministic nodes which 
are the time-driven sensor node and event-driven controller node are sharing the network in Figure 2. The 
HMM model proposed in Section 2 with 125 states for three random channels with the channel conditions 
{VL, L, M, H,VH]. An adaptive (HMM-1) is constructed by selecting an appropriate window size n = 10 and 
the Viterbi algorithm predicts future traffic volume for random channels. Since rest of the channels are 
deterministic and they have fixed network parameters, their traffic can be calculated from the parameters. 
The controller contains an event-driven task and every instances, a sample arrives over the network from the 
sensor node. The network traffic model used in the paper is offline, however since HMM model of the 
network traffics are modelled, if priory network traffic in the channels are known in the selected window size 
this model will work for online network traffic model also. 

To evaluate the effect of the number of traffic classes of the HMM model, we construct an HMM 
model with 27 states for three random channels with three conditions(L, M, H}, dubbed (HMM -2). The latter 
is used to predict the next random traffic volume for comparison to HMM- 1. 

The traffic in the controller to actuator channel is 100 pac/0.1sec. The controller computes a control 
signal after receiving the packet, then the packet that includes control signal is sent to the event-driven 
actuator node. After predicting the traffic of the random channel at time k + 1, we have values for all 
network traffic that is used for bandwidth allocation. The bandwidth allocation is t(k + 1) = [t*(k + 
1), t*, 100], where t* is to be calculated using the convex optimization of (8). 

After allocating bandwidth optimally, the sampling period for the time-driven channel is obtained by 
employing (9). The sampling period of the sensor to controller channel hj (k + 1) is calculated using (10). 
For a satisfactory output response, the sampling period of the time-driven channel is adjusted to eleminate 
channel congestion. 

At first, fixed network parameters without bandwidth allocation and sampling rate adjustment is 
used. Network congestion occurs as the traffic of some channels exceeds their capacity. Figure 3 shows that 
the closed-loop system response is oscillatory because network congestion causes packet loss and time delay. 

The available bandwidth is allocated depending on the demand of each channel. Bandwidth 
allocation for the random and the event-driven controller channel is achieved using l4 optimization. Because 
the total capacity of Network is limited, insufficient bandwith can be allocated for the time-driven sensor 
channel and network congestion may occur for the time-driven sensor channel. Its sampling rate 1s adjusted 
to satisfy its allocated bandwidth capacity subject to stability and performance constraints. 

Figure 4 indicates that the sampling period of the sensor channel stays in the interval [0,0.25] for 
both HMM-1 and HMM-2. The sampling period is assigned according to the allocated bandwidth for the 
time-driven sensor channel. The sampling period never exceeds the prescribed upper and lower bounds 
[0.6, 1.1] ms. Figure 5 and Figure 6 show that the bandwidth allocation using both HMM-1 and HMM-2 
provides satisfactory responses. However, Figure 6 shows that the response of HMM-2 includes sharp 
perturbations because of the large prediction error and limited sampling period interval. Figure 7 shows the 
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global network scheduling for the channels in NCS in the Truetime network simulator [24]. The network 
scheduling varies based on the dynamic sampling period assignment and bandwidth allocation. 


o Sampling schedule HMM-1 
* Sampling schedule HMM-2 





OCG CED CXUDIEIE X 0XD GOOD CaS OCTO CGE DEE GD OGDO «D» Wo O  GO»O CED Gee XDD 





Time 


Figure 4. Sampling period of the sensor channel 
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Figure 7. Channels scheduling for HMM-1 (high=running data, low-idle) 
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Figure 8 indicates bandwidth allocation percentage for each channel according to the network 
trafficin the three random channels in the time interval [0, 0.05] for both HMM-1 and HMM-2. Based on the 
predicted states, Figure 9 verifies that HMM-1 provides satisfactory bandwidth allocation and meets traffic 
demand better than HMM-2. 





























D HMM-1 
° The Percentage of Allocation for the Random Channel-1 * HMM-2 
8 m a | ^ Il. X m Va & ^» ^h x & | a aah m a m A ^ E è 
S 02 - a 9-3 wow wow ¥ 9 Q Vy - E ¥ ww Ww - wow Ww ¥ v ¥ 
E 0 | | | | | l | | | 
a 0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045 0.05 
Time 
° The Percentage of Allocation for the Random Channel-2 
E 0 2 F Li ð ¥ ^ | ~ * ^ : l a^ ^ ww 4 i 6 5 6 c ` a^ B ^ w w + LE t ^ .* + * 6 ‘ 6 ^ 5 ð | - w ^ * . " i 1 
z 0 | | | | | | Í | | 
~ 0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045 0.05 
Time 
2 The Percentage of Allocation for the Random Channel-3 
| | | T | | | | | 
$ 02^ ? è s ^ © ^ ) 8 9 3 B8B .9 8 8 8 © 9999 5 9 j E 9 9 9o 9«e€99".2999 * y 9 69 ~ e 9 y ) 
E 0 | | | | | l l l | 
a 0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045 0.05 
Time 





4 The Percentage of Allocation for the sensor channel 
8 * y * Ps 9 » T [ [ T * T I T * * 
^ | O na A ^ ^ “as Sy ” a A a ^ 
5 0.2 i . o i s o . ¥ "3 ~ 3 ¥ ¥ ° Ww wow w w ww * * i ) 
S o0 * | | + | | | | | | L 
E 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045 0.05 
Time 





The Percentage of Allocation for the controller channel 
| | | | | | | 
add etna MEA MARO OE BAD nena eae SEERE Md 





| | | | | | | | | 
0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045 0.05 
Time 


Percentage 
o 
N 


Figure 8. Percentage bandwidth allocation 
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Figure 9. Traffic prediction in random channel 


Figures 10, 11, 12 and Table 1 show the predicted random traffic for HMM-1 and HMM-2, 
respectively. The random traffic in the network is predicted by employing HMM and the Viterbi algorithm 
and used for bandwidth allocation. The figures show that, prediction errors for the states are lower for HMM- 
1. This provides more efficient bandwidth allocation and sampling period scheduling based on the allocation 
of bandwidth for the sensor channel. 

Figure 13 and Table 1 reveal that an adequate but small window size provides better state 
prediction. An excessive window size results in larger state prediction errors because of changes in network 
dynamics over time that make older data inapplicable. Increasing the number of network states that define the 
traffic condition reduces the prediction error. 
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Figure 11. Traffic state prediction in random channel (HMM-2) 
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Table 1. Comparison of prediction error for HMM-1 and HMM-2 for different window sizes 


Window size(n) 


5 traffic conditions HMM-1 


3 traffic conditions HMM-2 


10 12.323290 45.1066% 
15 21.5228% 59.6954% 
20 30.0000% 68.3673% 
23 38.256490 73.743696 
30 45.97194% 76.1856% 
60 76.4894% 83.5106% 
70 81.2603% 83.9785% 
80 82.3021% 83.260996 


5. CONCLUSION 


This paper provides a new approach for bandwidth allocation for NCS. An adaptive HMM with 5 
traffic classes is constructed for random traffic and the traffic is estimated utilizing the Viterbi algorithm. The 
HMM parameters are updated based on the traffic data for a window of specified width to account for 
changes in traffic conditions. |, Optimization method is employed to allocate bandwidth for the time-driven 
sensor channel in NCS. The sampling period of the time-driven sensor channels is adjucted to mitigate 
network traffic and avoid packet loss and delay but kept within bounds dictated by performance requirements 
and bandwidth limitations. We present conditions for the stability of the resulting time-varying system. A 
second HMM network with only 3 traffic classes, as is common in the literature, is also designed for 
comparison to the 5-class HMM. Simulation results where the 5-class HMM outperforms the 3-class HMM 
demonstrate the importance of having a sufficient number of traffic classes for the HMM. Results for 
different data window widths demonstrate the importance of selecting an appropriate window width to 
capture the current traffic dynamics of the network. Future work will apply the methodology to more 
complex NCS and exploit knowledge of random traffic statistics for bandwidth allocation. 
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